{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Installation\n",
    "! pip install smolagents\n",
    "# To install from source instead of the last release, comment the command above and uncomment the following one.\n",
    "# ! pip install git+https://github.com/huggingface/smolagents.git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用 OpenTelemetry 检查运行记录"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> [!TIP]\n",
    "> 如果您是初次构建Agent，建议先阅读 [Agent 入门指南](https://huggingface.co/docs/smolagents/main/zh/tutorials/../conceptual_guides/intro_agents) 和 [smolagents 导览](https://huggingface.co/docs/smolagents/main/zh/tutorials/../guided_tour)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 为什么需要记录Agent运行？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调试Agent运行过程具有挑战性。\n",
    "\n",
    "验证运行是否正常进行很困难，因为Agent的工作流程本身具有 [设计上的不可预测性](https://huggingface.co/docs/smolagents/main/zh/tutorials/../conceptual_guides/intro_agents)（如果可预测，直接使用传统代码即可）。\n",
    "\n",
    "检查运行记录同样困难：多步骤的Agent往往会快速在控制台生成大量日志，而大多数错误只是\"LLM 低级错误\"类型的问题，通常LLM会在后续步骤中通过生成更好的代码或工具调用来自我修正。\n",
    "\n",
    "因此，在生产环境中使用监控工具记录Agent运行过程，对于后续检查和分析至关重要！\n",
    "\n",
    "我们采用 [OpenTelemetry](https://opentelemetry.io/) 标准来实现Agent运行监控。\n",
    "\n",
    "这意味着您只需添加少量监控代码，即可在正常运行Agent时自动记录所有信息到监控平台。以下是在不同OpenTelemetry后端实现此功能的示例：\n",
    "\n",
    "在监控平台上的展示效果如下：\n",
    "\n",
    "<div class=\"flex justify-center\">\n",
    "    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/smolagents/inspect_run_phoenix.gif\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 Arize AI Phoenix 配置遥测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先安装必要的软件包。这里我们选择安装 [Arize AI 的 Phoenix](https://github.com/Arize-ai/phoenix) 作为日志收集和检查方案，您也可以使用其他兼容 OpenTelemetry 的平台来完成收集与检查工作。\n",
    "\n",
    "```shell\n",
    "pip install 'smolagents[telemetry]'\n",
    "```\n",
    "\n",
    "接着在后台运行日志收集器：\n",
    "\n",
    "```shell\n",
    "python -m phoenix.server.main serve\n",
    "```\n",
    "\n",
    "最后配置 `SmolagentsInstrumentor` 来追踪Agent活动，并将追踪数据发送至 Phoenix 默认端点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from phoenix.otel import register\n",
    "from openinference.instrumentation.smolagents import SmolagentsInstrumentor\n",
    "\n",
    "register()\n",
    "SmolagentsInstrumentor().instrument()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "完成上述配置后，即可正常运行您的Agent！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from smolagents import (\n",
    "    CodeAgent,\n",
    "    ToolCallingAgent,\n",
    "    WebSearchTool,\n",
    "    VisitWebpageTool,\n",
    "    InferenceClientModel,\n",
    ")\n",
    "\n",
    "model = InferenceClientModel()\n",
    "\n",
    "search_agent = ToolCallingAgent(\n",
    "    tools=[WebSearchTool(), VisitWebpageTool()],\n",
    "    model=model,\n",
    "    name=\"search_agent\",\n",
    "    description=\"This is an agent that can do web search.\",\n",
    ")\n",
    "\n",
    "manager_agent = CodeAgent(\n",
    "    tools=[],\n",
    "    model=model,\n",
    "    managed_agents=[search_agent],\n",
    ")\n",
    "manager_agent.run(\n",
    "    \"If the US keeps its 2024 growth rate, how many years will it take for the GDP to double?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voilà!\n",
    "\n",
    "此时访问 `http://0.0.0.0:6006/projects/` 即可查看运行记录：\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/smolagents/inspect_run_phoenix.png\">\n",
    "\n",
    "如图所示，CodeAgent 调用了其托管的 ToolCallingAgent（注：托管Agent也可以是另一个 CodeAgent）执行美国2024年经济增长率的网络搜索。托管Agent返回报告后，管理Agent根据结果计算出经济翻倍周期！是不是很智能？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 🪢 Langfuse 配置遥测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本部分演示如何通过 `SmolagentsInstrumentor` 使用 **Langfuse** 监控和调试 Hugging Face **smolagents**。\n",
    "\n",
    "> **Langfuse 是什么？** [Langfuse](https://langfuse.com) 是面向LLM工程的开源平台，提供AI Agent的追踪与监控功能，帮助开发者调试、分析和优化产品。该平台通过原生集成、OpenTelemetry 和 SDKs 与各类工具框架对接。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤 1: 安装依赖"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install langfuse 'smolagents[telemetry]' openinference-instrumentation-smolagents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤 2: 配置环境变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设置 Langfuse API 密钥，并配置 OpenTelemetry 端点将追踪数据发送至 Langfuse。通过注册 [Langfuse Cloud](https://cloud.langfuse.com) 或 [自托管 Langfuse](https://langfuse.com/self-hosting) 获取 API 密钥。\n",
    "\n",
    "同时需添加 [Hugging Face 令牌](https://huggingface.co/settings/tokens) (`HF_TOKEN`) 作为环境变量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "# Get keys for your project from the project settings page: https://cloud.langfuse.com\n",
    "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"pk-lf-...\" \n",
    "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"sk-lf-...\" \n",
    "os.environ[\"LANGFUSE_HOST\"] = \"https://cloud.langfuse.com\" # 🇪🇺 EU region\n",
    "# os.environ[\"LANGFUSE_HOST\"] = \"https://us.cloud.langfuse.com\" # 🇺🇸 US region\n",
    " \n",
    "# your Hugging Face token\n",
    "os.environ[\"HF_TOKEN\"] = \"hf_...\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langfuse import get_client\n",
    " \n",
    "langfuse = get_client()\n",
    " \n",
    "# Verify connection\n",
    "if langfuse.auth_check():\n",
    "    print(\"Langfuse client is authenticated and ready!\")\n",
    "else:\n",
    "    print(\"Authentication failed. Please check your credentials and host.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤 3: 初始化 `SmolagentsInstrumentor`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在应用程序代码执行前初始化 `SmolagentsInstrumentor`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openinference.instrumentation.smolagents import SmolagentsInstrumentor\n",
    " \n",
    "SmolagentsInstrumentor().instrument()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤 4: 运行 smolagent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from smolagents import (\n",
    "    CodeAgent,\n",
    "    ToolCallingAgent,\n",
    "    WebSearchTool,\n",
    "    VisitWebpageTool,\n",
    "    InferenceClientModel,\n",
    ")\n",
    "\n",
    "model = InferenceClientModel(\n",
    "    model_id=\"deepseek-ai/DeepSeek-R1-Distill-Qwen-32B\"\n",
    ")\n",
    "\n",
    "search_agent = ToolCallingAgent(\n",
    "    tools=[WebSearchTool(), VisitWebpageTool()],\n",
    "    model=model,\n",
    "    name=\"search_agent\",\n",
    "    description=\"This is an agent that can do web search.\",\n",
    ")\n",
    "\n",
    "manager_agent = CodeAgent(\n",
    "    tools=[],\n",
    "    model=model,\n",
    "    managed_agents=[search_agent],\n",
    ")\n",
    "manager_agent.run(\n",
    "    \"How can Langfuse be used to monitor and improve the reasoning and decision-making of smolagents when they execute multi-step tasks, like dynamically adjusting a recipe based on user feedback or available ingredients?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤 5: 在 Langfuse 中查看追踪记录"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行Agent后，您可以在 [Langfuse](https://cloud.langfuse.com) 平台查看 smolagents 应用生成的追踪记录。这些记录会详细展示LLM的交互步骤，帮助您调试和优化AI代理。\n",
    "\n",
    "![smolagents 追踪示例](https://langfuse.com/images/cookbook/integration-smolagents/smolagent_example_trace.png)\n",
    "\n",
    "_[Langfuse 公开示例追踪](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/ce5160f9bfd5a6cd63b07d2bfcec6f54?timestamp=2025-02-11T09%3A25%3A45.163Z&display=details)_"
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 4
}
